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INTRODUCTION 


This paper presents an overview of the types of problems that are being considered by control 
theorists in the area of dynamic large-scale systems with emphasis on decentralized control strate- 
gies. A similar paper by Varaiya (ref. 1) indicated the interplay between static notions drawn from 
the mathematical economics, management, and programming areas and the attempts by control 
theorists to extend the static notions into the stochastic dynamic case. In this paper, we shall not 
elaborate upon the dynamic or team aspects of large-scale systems. Rather we shall concentrate on 
approaches that deal directly with decentralized decision making for large-scale systems. 

Although a survey paper, the number of surveyed results is relatively small. This is due to the 
fact that there is still not a unified theory for decentralized control. What is available is a set of 
individual contributions that point out both “blind alleys” as well as potentially fruitful 
approaches. 

What we shall attempt to point out is that future advances in decentralized system theory are 
intimately connected with advances in the soK:alIed stochastic control problem with nonclassical 
information pattern. To appreciate how this problem differs from the classical stochastic control 
problem, it is useful to briefly summarize the basic assumptions and mathematical tools associated 
with the latter. This is done in section 2. Section 3 is concerned with certain pitfalls that arise when 
one atterripts to impose a decentralized structure at the start, but the mathematics “wipes out” the 
original intent. Hence, one can draw certain conclusions about the proper mathematical formulation 
of decentralized control problems. Section 4 surveys some research (primarily carried out by the 
author and his students) that attempts to circumvent some of the pitfalls discussed in sections. 
Section 5 presents some conclusions about future research. 

CLASSICAL STOCHASTIC CONTROL PROBLEM 


This section reviews in an informal way the classical stochastic control problem or the problem 
of stochastic control with classical information structure. Our main purpose is to indicate the types 
of assumptions one makes in this class of problems, the nature of the mathematical tools available, 
and the general structure of the solution. This overview is necessary so that one can see that the 
solution to the classical stochastic control problem leads to a completely centralized system. 
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From a technical point of view, the best survey of the issues associated with classical stochastic 
control is, in the opinion of the author, the paper by Witsenhausen (ref. 2). The classical stochastic 
control problem is the subject of many standard texts and monographs (see, e.g., refs. 3 to 7). 

Figure 1 is an abstract block diagram of a centralized control system. One deals with a 
stochastic dynamical system (usually described by a set of stochastic difference or differential 
equations) and one makes possibly noisy measurements of certain of its variables. The time evolu- 
tion of the system variables is influenced by stochastic disturbances and decisions (or control) 
variables generated in closed-loop feedback form by a single controller or decision maker. 

The assumptions made in the classical stochastic control problem are as follows: 

A2.1 There is a single controller or decision maker. 

A2.2 The controller knows the mathematical form of the system dynamics (i.e., the stochastic 
differential or difference equations). 

A2.3 The controller knows the relationship of the measurements to the system variables. 

A2.4 The controller knows a priori the probability densities of all underlying stochastic varia- 
bles (i.e., exogenous disturbances, uncertain system parameters, measurement errors, etc.). 

A2.5 The controller wishes to minimize a well-defined scalar deterministic cost functional. 

A2.6 The controller at any time t has instant and perfect recall of all past applied inputs or 
decisions and all past and present measurements. 

Under these assumptions, classical stochastic control provides a well-defined rule that translates all 
information available to the controller at time t (i.e., the contents of assumptions A2.1 to A2.6) to 
an optimal control or decision at time t. 

From a technical point of view the state variable (causal) description of the dynamical system, 
the use of a Bayesian rule to deal with the stochastic elements, and the use of stochastic dynamic 
programming blend well to yield the optimal stochastic control in a relatively straightforward 
conceptual manner. Actual calculations are generally very complex with the exception of the 
so-called Linear-Quadratic-Gaussian (LQG) problem (refs. 4-8). 


DECENTRALIZED CONTROL: PITFALLS 


This section presents the types of issues that arise when the basic assumptions of section 2 
associated with classical centralized stochastic control are modified. There are several ways to 
depart from the basic framework of classical stochastic control. In this paper, we shall adopt the 
viewpoint of examining the issues when we wish to analyze some sort of hierarchical multilevel 
decentralized system (ref. 9). 
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One does not have to examine a complex hierarchical structure to understand the issues 
associated with decentralized control. Figure 2 presents the simplest possible case involving a 
two-level structure. We shall elaborate upon the structure implied in figure 2 to point out its general 
characteristics. 


Case 3.1 

Imagine for the time being that the “interaction” channel and the “coordinator” did not 
appear in figure 2. We are left with two “uncoupled” dynamical systems. If the framework of 
section 2 is adopted, we can postulate that each controller solves a classical stochastic control 
system. 


Case 3 .2 

Next, let us still leave the “coordinator” out of figure 2, but restore the “interaction” channel. 
What we mean by “interaction” is that certain decision and/or state and/or output variables of each 
system influence the dynamic evolution of the other system. If this interaction is “weak,” then it is 
possible for both systems to operate non-optimally but still satisfactorily without altering the basic 
control strategy of Case 3.1, because the interactions are viewed as exogenous unknown disturb- 
ances and the inherent use of feedback tends to make the overall system response somewhat 
insensitive to weak, unmodeled disturbances. 

This situation, namely, with weak interaction and the absence of coordination, has been 
analyzed by Chong, Kwong, and Athans (refs. 10-12). This research attempted to replace the weak 
interaction disturbances, which are actually correlated in time, with equivalent “fake white noise” 
inputs which are uncorrelated in time, and to evaluate techniques by which, in the LQG context, 
the convariance of the “fake white noise” can be selected. 


Case 3 .3 

If the dynamic interaction between the systems is not negligible, then the performance of each 
system in figure 2 can be expected to deteriorate severely. To “cure” this performance degradation, 
one introduces the “coordinator” in figure 2. 

Intuitively, in any physical large-scale system, the role of the coordinator is to receive some 
sort of information from the local subsystems and make some decisions to improve the performance 
compared to that under Case 3.2. The crucial question then is to make precise the role of the 
coordinator as a function of postulated strategies for the lower level subsystems. 

It is possible, in any given physical situation, to specify the task of the coordinator in a 
reasonably good, but ad hoc way, so that the overall system performance is satisfactory for the 
specific application. However, the heart of decentralized control theory research is to formulate 
precise mathematical problems whose solution defines the optimal task of the coordinator, without 
destroying the intuitively appealing decentralized structure in figure 2. 


103 



Some Blind Alleys 


The following assumptions sound reasonable from a physical point of view, but when they are 
incorporated in a mathematical framework, the mathematical solution destroys the decentralized 
structure. The pitfalls that the assumptions lead to are easily seen without resorting to complex 
mathematics, and the appreciation of the pitfalls provides valuable knowledge on how not to 
formulate a decentralized control problem. 

We start with a list of assumptions, again keeping the structure of figure 2 in mind: 

A3.1 Each local system neglects the interaction from the other system. 

A3. 2 Under A3.1, each local controller knows the dynamics and probabilistic information 
associated with his own system, has his own performance index, and has perfect recall of his own 
past measurements and controls. It follows (see section 2) that each local system can solve its own 
well-formulated classical stochastic control problem, and we assume that each local system applies 
in real time the optimal stochastic control obtained under these assumptions. 

A3. 3 The coordinator knows the dynamics of both subsystems, including the interaction, as 
well as all prior probabilistic information available to each local subsystem. 

A3 .4 The coordinator’s cost functional (performance index, utility function) is a well-defined 
function of the cost functionals of each local subsystem (e.g., a weighted sum). 

A3. 5 At each instant of time, the coordinator can apply a dynamic control to each local 
system of the same nature of the local control. 

A3 .6 At each instant of time, each local subsystem transmits instantly and without error its 
measurements and controls to the coordinator; furthermore, the coordinator has perfect recall. 

The key question is then: Under assumptions A3.1 to A3.6, what is the optimal decision rule 
for the coordinator? The answer is exceedingly simple. Under assumptions A3 .3 to A3 .6, the 
coordinator has a classical stochastic control problem for the entire system. Hence, so far as the 
coordinator is concerned, he must solve the overall optimal stochastic control problem (see section 
2) and his optimal strategy is to (i) cancel the locally computed controls (see assumption A3 .2) and 
(ii) substitute the global optimal controls. 

Thus, the essential decentralized nature of the problem is destroyed. This points out that, even 
in the stochastic case, one cannot allow the coordinator full knowledge of everything because the 
mathematically optimal solution allows the coordinator to completely take over. This problem is 
even more serious if a complete deterministic framework for decentralized control is adopted 
(ref. 9). 


Control-Sharing Strategies 

So that the coordinator does not take over completely, some of assumptions A3.1 to A3 .6 
must be modified to deny the coordinator full knowledge of everything. Needless to say, there are 
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many ways to modify assumptions A3.1 to A3 .6 and to attempt an analysis of the role of the 
coordinator. 


In this section, we shall examine one variation because it has received some attention in the 
control literature, although not precisely in the context of this paper. Hence our remarks represent 
a reinterpretation of the research of Aoki (ref. 1 3) and SandeU and Athans (ref. 14). 

One can argue that the coordinator can take over under assumptions A3.1 to A3 .6 because 
assumption A3 .6 allows the coordinator to have instantaneous access to all measurements of the 
local systems. This allows him (see assumption A3 .3) to calculate the local controls to be used by 
the subsystems (see assumption A3 .2) and cancel them (see assumption A3. 5). Hence one may 
think that one way to prevent the coordinator from taking over is to deny to him the actual 
measurements of the local systems. Thus, we seek to modify assumption A3 .6. 


Assumption A3 .6, however, deals not only with measurements but with controls. We can 
adopt the intuitive philosophy “do not flood your boss with day-to-day occurrences, but let him 
know your day-to-day decisions.” If we adopt this framework, we can replace assumption A3 .6 
with the following: 


A3.6(M) At each instant of time, each local subsystem transmits instantly and without error 
ONLY its controls, but not its measurements to the coordinator; furthermore, the coordinator has 
perfect recall. 

One can then pose a mathematical problem under assumptions A3.1 to A3 .5 and A3.6(M) to 
find the optimal decision rule for the coordinator. The answer (refs. 13, 14) is both surprising and 
interesting: (i) the stochastic control problem for the coordinator is not well defined, in the sense 
that an optimal solution does not exist; and (ii) although an optimal solution does not exist, one 
can find e optimal solutions in the sense that one can approach the unattainable optimal solution 
arbitrarily closely. 

The way these e optimal solutions are obtained is interesting and instructive because they 
indicate once more how not to formulate a decentralized control problem. We shall attempt to 
explain how this happens by a simple example. 


From figure 2, let us suppose that the system operates in discrete time so that measurements 
and decisions are made at the values of the time index t = 0, 1, 2, 3, . . . . LetZi(t) denote the 
measurement and let Ui(t) denote the control of system 1 ; for simplicity, assume that both Z\{.t) 
and u^(t) are scalars. Suppose that the following sequence of measurements has been made by 
system 1 : 


t = 0, z (0) = 6 
/ - I, z(l) = 7 
t = 2, z (2) = 8 
t = 3, z (3) = 9 


( 1 ) 
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Under assumption A3.2, system 1 has a well-defined rule for generating its own optimal control. 
Suppose that, at the basis of the measurements of equation (1), the optimal local control for 
system 1 at the time / = 3 is 


u,* (3) = 1.234 . (2) 

Under assumption A3.6(M), the control in equation (2) can be transmitted instantly and without 
error to the coordinator. However, the nature of the e optimal solution indicates that system 1 
should not transmit the control (eq. 2) to the coordinator. Rather, it should transmit and apply to 
the system a control of the following form: 

ui O) = 1.234000000 . . . 00006789. (3) 

The information conveyed to the coordinator when he receives the control (eq. 3) without error is 
very different from that contained in equation (2). Examination of equations (1) to (3) indicates 
that the past measurements (6, 7, 8, 9) have been coded in Mi( 3). From the front part of equa- 
tion (3), the coordinator knows the control Mi*( 3) of equation (2); from the tail end of equa- 
tion (3), the coordinator knows exactly the past measurements of system 1 . The string of zeros 
between the control (1.234) and the coded measurements (6789) is simply to guarantee that the 
application of Ui(3), rather than Uj *(3), to the system results in an infinitesimal loss in system 
performance (i.e., the 000 . . . 006789 part of the control is wiped out by the system uncertainty). 

Hence, assumption A3.6(M) is not strong enough to prevent the coordinator from obtaining all 
the information he needs to take over for all practical cases. 


Conclusions 

The above discussion points out that, in stochastic decentralized problems, instantaneous 
error-free transmissions of either both controls and measurements or controls alone is not a realistic 
mathematical assumption because this allows the coordinator to take over and destroy the decentra- 
lized nature of the problem. 


DECENTRALIZED CONTROL: PROMISING AVENUES 


In this section, we discuss some recent results that appear to be useful toward building some 
elements of a theory for decentralized control. Once more the reader is referred to Variaya (ref. 1) 
for additional concepts and discussion. 


Decentralization with Fixed Structure 

Figure 3 depicts a specific decentralized structure somewhat different from that discussed in 
section 3. One is given an nth order, linear stochastic dynamic system, with two sets of measure- 
ments ( z 1 and Zj ) and two sets of controls ( u , and ). It is decided a priori to select two dynamic 
controllers that generate stochastic controls in the manner illustrated in figure 3. It is assumed that 
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there is a single cost functional to be minimized, and one can view the job of the coordinator as 
defining the characteristics of the two controllers. This represents a variation on the dynamic team 
problem (see, e.g., Ho and Chu (refs. 15, 16)). 

Because of the nonclassical information pattern and the general lack of knowledge -for solving 
dynamic team problems, some additional assumptions have to be made so that the problem of 
designing the two controllers in figure 3 can be solved. 

For the LQG continuous-time case, Chong and Athans (refs. 10, 17) fixed the structure of 
each controller to be linear and of the same dimension as the order of the dynamic system that was 
controlled. Furthermore, each controller was constrained so that its internal Kalman-Bucy filter 
would produce unconditional zero mean estimates of the state, ignoring the actions of the other 
controller. The parameter matrices of each dynamic controller could then be globally optimized by 
solving a deterministic matrix optimal control problem through the use of the matrix minimum 
principle (ref 18). The discrete-time version of this problem was considered by Carpenter (ref. 19). 

Two basic conclusions can be drawn from the above studies (see also Variaya (ref. 1)): 

(i) The off-line computational effort for solving such decentralized problems is greater than 
that required for the centralized case. 

(ii) Even in the LQG context, the separation theorem or certainty equivalence principle fails 
to hold. 


Periodic Coordination 

1 

The discussion in section 3 indicates that for decentralized systems (fig. 3), one cannot provide 
the coordinator with instantaneous and error-free transmission of the local subsystem measurements 
and/or decisions to the coordinator; otherwise, the coordinator takes over and substitutes the 
globally optimal stochastic controls, thus overriding the decisions of the local controller. 

One way to bypass this problem is to assume that the coordinator is allowed to “interfere” 
only occasionally. This notion of periodic coordination has been considered by Chong and Athans 
(refs. 20-22). To understand the intuitive notion of periodic coordination, suppose that assumptions 
A3.1 to A3.4 of section 3 are still valid, but assumptions A3. 5 and A3.6 are replaced by the 
following (informal) one. 

Periodic coordination structure - Suppose that the entire system operates in discrete time. For 
concreteness, we assume that the basic time unit is a day. Then the basic system operation is 

(i) Assume that each lower-level system makes its measurements and generates its controls 
(decisions) once a day. 

(ii) Once a month, all lower-level system measurements and controls are “mailed” without 
error to the coordinator. 
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The basic question is: what is the job of the coordinator at the beginning of each month? The 
mathematical approach adopted and the results obtained (refs. 20-22) have the following interpreta- 
tion which we feel has certain intuitively appealing aspects. 

Once a month, the coordinator has a threefold task with respect to each lower-level system: 

(i) Set it straight. In a technical context, he corrects the estimates generated by the lower-level 
Kalman filters because these estimates are in error because each lower-level system neglects the 
interactions from the other lower-level systems. 

(ii) Change its directives, in the sense that new time paths for the lower-level controls are given 
in an open-loop sense. 

(iii) Change its incentives, in the sense that additional terms are added to each lower-level 
system cost functional to compensate for the fact that the global cost functional differs from each 
lower-level cost functional. 

The main advantage of these results is that the mathematical theory itself suggests the tasks that 
must be performed periodically by the coordinator. The main disadvantage is that the coordinator 
must still solve a very large-scale stochastic optimization problem, although not as often as in the 
basic time frame of the lower level. Although for certain applications this approach may be feasible, 
it lacks the capability of somehow aggregating the information flow from the lower-level systems to 
the coordinator. 

Nonetheless, because the theory itself suggests this mode of coordination (by changing direc- 
tives and incentives), it provides strong motivation to postulate a specific framework for operating 
the lower-level systems. A preliminary formulation along these lines can be found in a recent paper 
by Athans (ref. 23). i 

The notion of delaying the information exchanged between different portions of a hierarchical 
system is intuitively appealing. Much more research is needed to understand its impact on decentra- 
lized control theory. However, the results of Sandell and Athans (ref. 14), in which it was shown 
that LQG problems with a unit-time step delay of information exchange admit a linear optimal 
decision rule, which can be calculated explicitly, appear to be promising so far as their applicability 
to decentralized control theory is concerned. 


Remarks 

Most of the results surveyed attempt in one way or another to present to both the coordinator 
and the local subsystems a classical stochastic control problem. Although research along these lines 
is useful, there is no definitive theory that deals directly with issues of aggregated information, 
decision making with partial information, or decision making with finite memory. 

To adequately deal with these issues in the context of decentralized control, much additional 
research is needed in the area of stochastic control with nonclassical information patterns. The 
famous Witsenhausen counter-example (ref. 24), in which a simple LQG problem with nonclassical 
information pattern was shown to have a nonlinear optimal decision rule, points out the immense 
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difficulties associated with this class of problems. Witsenhausen (refs. 25, 26) has continued his 
fundamental investigations in this class of problems, but their implications in the context of 
decentralized control theory remain largely unexplored. The work on finite-state, finite-memory 
control of Sandell and Athans (refs. 27, 28) may be useful to aggregate the flow of information 
between the different levels of a hierarchy and to limit the computational complexity available to 
the coordinator. In addition, the recent results surveyed by Ho (ref. 29) pertaining to approaches in 
information structures when many decision makers are involved is of direct importance to 
decentralized control problems. 


CONCLUSIONS 


The main conclusions that one can draw are: 

(1) Any purely deterministic approach to multilevel hierarchical dynamic systems is not apt to 
lead to realistic theories or designs. 

(2) The flow of measurements and decisions in a decentralized system should not be 
instantaneous and error-free. 

(3) Delays in information exchange in a decentralized system lead to reasonable approaches to 
decentralized control. 

(4) A mathematically precise notion of aggregating information is not yet available. 

(5) Research in nonclassical information structures is directly relevant to problems of 
decentralized control. 
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Figure L— Centralized control system. 



Figure 2.— Two level hierarchical structure. Figure 3.— Decentralized structure. 
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